MIT Develops AR System for 3D Ultrasound Imaging
Researchers at the Massachusetts Institute of Technology (MIT) have developed a new ultrasound imaging system that combines three-dimensional imaging with augmented reality, allowing users to view a real-time 3D representation of the object being scanned.
Medical ultrasound imaging typically produces two-dimensional images. Interpreting those images requires technicians and healthcare professionals to mentally reconstruct a three-dimensional view of the tissue or structure being examined. The process requires specialised training and can be difficult to master.
The MIT research team developed a new approach to reduce the cognitive effort required to interpret ultrasound images. The system enables users wearing an augmented reality or virtual reality headset to view a precise digital representation of the scanned object in three dimensions. The image is overlaid on the object’s physical location, allowing users to observe the internal structure from different angles.
According to the researchers, the technology could support the training of ultrasound technicians and other healthcare professionals who use ultrasound imaging. It could also be used in hospitals for procedures that call for precise image guidance, including needle placement during biopsies.
The study was published in Nature Communications Engineering. The senior author was Canan Dagdeviren, an associate professor of media arts and sciences at MIT. Lead authors were MIT graduate students Jason Hou and Shrihari Viswanath. Other contributors included Bowen Wu, Cinay Dilibal and Tanisha Shende.
Ultrasound imaging works by transmitting high-frequency acoustic waves into the body. These waves reflect from tissues and return to an ultrasound transducer. The transducer converts the returning signals into electrical data, which is then used to generate two-dimensional images. Healthcare professionals trained in ultrasound interpretation learn to convert these images into a mental three-dimensional representation of the anatomy being examined.
The researchers reported that this process creates a significant cognitive burden. Interpreting multiple image slices and reconstructing them into a three-dimensional structure can demand considerable training and experience. According to the research team, this challenge can contribute to mistakes during scanning.
To address this issue, the researchers combined augmented reality technology with three-dimensional ultrasound imaging.
Three-dimensional ultrasound imaging is already used in some medical fields, including foetal imaging and echocardiography. However, the researchers noted that many existing three-dimensional ultrasound systems are expensive and are not widely available. For the study, the team used a real-time three-dimensional ultrasound system previously developed for breast cancer detection.
The new system includes an ultrasound probe that is slightly smaller than a pack of playing cards. The probe transmits information using a chirped data acquisition system, known as cDAQ. It contains an ultrasound array arranged in the shape of an empty square. This arrangement enables the probe to capture three-dimensional images of the underlying tissue.
The researchers reported that the system uses fewer ultrasound elements than a conventional three-dimensional ultrasound system. As a result, it requires less power and can be built at a lower cost.
Data collected by the probe is compressed and streamed into Unreal Engine, a three-dimensional computer graphics platform. The software converts voxel data from the ultrasound scan into a direct three-dimensional representation of the scanned object without information loss.
Users wearing an augmented reality or virtual reality headset can then view the resulting image positioned over the scanned object. By changing their viewing angle or moving closer to the object, users can observe different perspectives of the structure being examined. The researchers stated that this makes it easier to identify and analyse the scanned object.
The system was named AR-VIU, which stands for augmented real-time volumetric imaging in ultrasound.
To evaluate the technology, the researchers conducted tests involving 18 participants. Nine participants were experienced ultrasound users, including physicians and sonographers. The remaining nine participants had no previous experience with ultrasound systems.
Each participant completed diagnostic tasks using four different ultrasound imaging approaches. One method involved viewing conventional two-dimensional ultrasound images on a standard screen, which reflects how many ultrasound examinations are currently performed. Participants also used a three-dimensional imaging system displayed on a standard screen. In addition, they tested a two-dimensional augmented reality system and the AR-VIU three-dimensional augmented reality system.
In one experiment, participants were asked to identify objects embedded within gelatin inside opaque containers. The objects included a spring, a ball or a screw. The containers were scanned using ultrasound imaging, and participants attempted to determine which object was present.
In a second experiment, participants were asked to identify and mark the location of tissue phantom material. A tissue phantom is a gel-like material engineered to mimic human tissue. The task was designed to simulate procedures that require accurate needle placement, such as biopsies.
The researchers found that the AR-VIU system improved participants’ ability to identify and locate objects. According to the study, the improvement was seen across all users. The effect was particularly strong among participants with no previous ultrasound experience.
The research team reported that novice users performed nearly as well as experienced users when using AR-VIU. By contrast, experienced users performed considerably better than novices when using the conventional two-dimensional imaging system.
The researchers also conducted interviews following the experiments. Most novice participants reported preferring the AR-VIU system and indicated that it made the tasks easier to perform and understand.
Experienced ultrasound users generally preferred the traditional two-dimensional imaging system because it was the approach with which they had been trained and were most familiar. However, the researchers reported that many experts also identified possible advantages for the AR-VIU system in specific clinical situations. These included guiding needle placement during biopsies and visualising heart wall movement during echocardiography.
The research team is continuing to improve the technology. Current work includes efforts to improve image resolution and further testing intended to evaluate the accuracy of the AR-VIU system.
The research was funded by the MIT Media Lab Consortium, the National Science Foundation, an MIT HEALS graduate fellowship and an MIT-Tata graduate fellowship.
